Skip to content

Exit cleanly when startup fails instead of leaving a zombie JVM (fixes #498, fixes #499) - #565

Open
iacsha wants to merge 1 commit into
OpenIntegrationEngine:mainfrom
iacsha:fix/498-499-exit-on-startup-failure
Open

iacsha wants to merge 1 commit into
OpenIntegrationEngine:mainfrom
iacsha:fix/498-499-exit-on-startup-failure

Conversation

@iacsha

@iacsha iacsha commented Nov 5, 2026

Copy link
Copy Markdown

Exit cleanly when startup fails instead of leaving a zombie JVM (fixes #498, fixes #499)

Problem

MirthLauncher loads com.mirth.connect.server.Mirth as a Thread and starts it; it never
calls Mirth.main(), so main()'s System.exit(0) never runs in the real launcher path.
When run() aborts a failed startup it simply returns — the Mirth thread dies, but the
launcher's non-daemon threads keep the JVM alive, so the process stays up with no listening
ports. A service manager sees it as running; nothing restarts or alerts (#498).

Two confirmed triggers:

  • A port already in use (e.g. a second instance, or another service on 8443).
  • A failed database migration (MigrationException).

On the migration path there is a second problem (#499): the shutdown hook, registered before
startup(), runs the full Mirth.shutdown(), which reads the database while stopping the
engine. Startup already called stopDatabase(), so on Derby the hook waits for a pooled
connection that never comes and hangs — the process must be SIGKILLed.

Fix

  • System.exit(EXIT_STARTUP_FAILURE) on both abort paths (port-in-use and MigrationException)
    so the JVM actually terminates with a non-zero status a service manager can see.
  • Track startupComplete; the ShutdownHook runs the full shutdown() only when startup
    completed. Exiting after a failed migration therefore skips the engine/database shutdown that
    would hang on Derby.

Normal operation is unchanged: a server that finished starting still shuts down fully via the
hook and via the SHUTDOWN_SERVER command.

Verification

Built from this branch and run live (Open Integration Engine 4.7.0, embedded Derby, in Docker):

Port already in use — start a second instance while 8443 is taken:

  • Before: the second JVM stayed alive (orphan process, no listeners); the wrapper had to be timed out.
  • After: exits code 1 in ~2s, no orphan process.

Failed migration — set SCHEMA_INFO.VERSION to an unknown value, then start:

  • After: logs MigrationException: Database schema version "99.0.0" is not recognized ... and the
    process exits code 1 in ~5s (no Derby shutdown hang, no SIGKILL needed).

(The pre-fix port-in-use zombie was reproduced directly; the migration-path Derby hang matches the
behavior documented in #499 and shares the same root cause.)

@github-actions

github-actions Bot commented Nov 6, 2026

Copy link
Copy Markdown

Test Results

  141 files  ±0    141 suites  ±0   3m 33s ⏱️ +6s
  919 tests ±0    919 ✅ ±0  0 💤 ±0  0 ❌ ±0 
1 051 runs  ±0  1 045 ✅ ±0  6 💤 ±0  0 ❌ ±0 

Results for commit adaef4c. ± Comparison against base commit 991ed69.

…OpenIntegrationEngine#498, fixes OpenIntegrationEngine#499)

MirthLauncher runs Mirth as a Thread and never calls main(), so main() System.exit(0) never runs; an aborted startup just returned and left the JVM alive with no listeners. Exit on the port-in-use and MigrationException abort paths, and guard the shutdown hook with a startupComplete flag so exiting after a failed migration does not run the full shutdown (which hangs reading the stopped Derby database).

Signed-off-by: iacsha <122627209+iacsha@users.noreply.github.com>
@iacsha
iacsha force-pushed the fix/498-499-exit-on-startup-failure branch from adaef4c to 25d059d Compare November 6, 2026 19:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Stopping the server after a failed startup hangs on Derby Server process keeps running after a startup failure

1 participant